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INTRODUCTION 


The  enclosed  summaries  provide  an  outline  of  some  of  the  URI  research  projects.  In 
addition  to  these  activities,  a  scientific  meeting  combining  URI  scientists  and  other  dis¬ 
tinguished  vision  researchers  in  the  USA  and  Canada  was  organized  and  held  at  Boston 
University  in  March,  1988.  Members  of  the  URI  also  played  a  key  role  in  founding  the 
International  Neural  Network  Society  (INNS)  and  organizing  its  first  annual  meeting  in 
Boston  on  September  5-9,  1988,  where  600  papers  were  presented  to  1700  participants. 
Professor  Grossberg  was  INNS  President  and  Professor  Carpenter  was  the  meeting’s  orga¬ 
nization  chairman.  Professors  Carpenter,  Cohen,  Daugman,  Grossberg,  and  Mingolla  gave 
featured  lectures.  Many  additional  scientific  interactions  among  URI  participants — such 
as  between  Arend  and  Reeves;  Arend,  Grossberg,  and  Todorovie;  Grossberg,  Reeves,  and 
Rudd;  Mingolla  and  Todd;  Arend,  Daugman,  Mingolla,  and  Savoy;  Richards  and  Searle; 
Daugman  and  Grossberg — we™  undertaken. 


PUBLICATIONS  PARTIALLY  SUPPORTED  BY 
THE  AIR  FORCE  OFFICE  OF  SCIENTIFIC  RESEARCH 
AT  THE  CENTER  FOR  ADAPTIVE  SYSTEMS 


CONTRACT  AFOSR  F49620-87-C-0018 
NOVEMBER  1,  1986-SEPTEMBER  SO,  1988 


1.  Banquet,  J.-P.  and  Grossberg,  S.  (1987).  Probing  cognitive  processes  through  the  struc¬ 
ture  of  event-related  potentials  during  learning;,  An  experimental  and  theoretical  analysis. 
Applied  Optics,  26,  4931-4946.  (+)  - 


2.  Bullock,  D.  and  Grossberg,  S.  (1987]rwA  neural  network  architecture  for  automatic  tra¬ 
jectory  formation  and  coordination  of  multiple  effectors  during  variable-speed  arm  move- 
mentSj  Li  M.  Caudill  and  C.  Butler  (Eds.),  Proceedings  of  the  IEEE  international  confer¬ 
ence  on  neural-networks,  TV,  559-566.  (+) 


3.  Bullock,  D.  and  Grossberg,  S.  (1988)  .^Neural  dynamics  of  planned  arm  movements:  Emer¬ 
gent  invariants  and  speed-accuracy  properties  during  trajectory  formation,  Psychological 
Review,  95,  49-90.  (+) 


4.  Bullock,  D.  and  Grossberg,  S.  (1988).  Seif-organizing  neural  architectures  for  eye  move¬ 
ments,  arm  movements,  and  eye-arm  coordination  In  H.  Haken  (Ed.),  Proceedings  of 
the  international  workshop  on  neural  and  synergetic  computers,  Schloss  Elmau, 
Bavaria.  (+)*.’■''.  /  , 


rvo . 


5.  Carpenter,  G.A.  and  Grossberg,  S.  (1987).  ART  2:  Self-organization  of  stable  cate¬ 
gory  recognition  codes  for  analog  input  patterns.  In  M.  Caudill  and  C.  Butler  (Eds.), 
Proceedings  of  the  IEEE  international  conference  on  neural  networks,  H,  727-736.  (+*) 


6.  Carpenter,  G.A.  and  Grossberg,  S.  (1987).  Invariant  pattern  recognition  and  recall  by  an 
attentive  self-organizing  ART  architecture  in  a  nonstationary  world.  In  M.  Caudill  and  C. 
Butler  (Eds.),  Proceedings  of  the  IEEE  international  conference  on  neural  networks,  H, 
737-746.  (+*) 

7.  Carpenter,  G.A.  and  Grossberg,  S.  (1987).  A  massively  parallel  architecture  for  a  self¬ 
organizing  neural  pattern  recognition  machine.  Computer  Vision,  Graphics,  and  Image 
Processing,  37,  54-115.  (+*) 


8.  Carpenter,  G.A.  and  Grossberg,  S.  (1987).  ART  2:  Self-organization  of  stable  category 
recognition  codes  for  analog  input  patterns.  Applied  Optics,  26,  4919-4930.  (+*) 

9.  Carpenter,  G.A.  and  Groasberg,  S.  (1988).  Th*:  ART  of  rdaptive  pattern  recognition  by 
a  self-organizing  neural  network.  Computer,  21,  77-88.  (+) 

10.  Carpenter,  G.A.  and  Grossberg,  S.  (1988).  Self-organizing  neural  network  architectures 
for  real-time  adaptive  pattern  recognition.  In  H.  Haken  (Ed.),  Proceedings  of  the  inter¬ 
national  workshop  on  neural  and  synergetic  computers,  Schloss  Elmau,  Bavaria. 

(+*) 


11.  Cohen,  M.A.  (1987).  Sustained  oscillations  in  symmetric  cooperative-competitive  neural 
networks.  In  M.  Caudill  and  C.  Butler  (Eds.),  Proceedings  of  the  IEEE  international 
conference  on  neural  networks ,  II,  127-130.  (+) 

12.  Cohen,  M.A.  and  Grossberg,  S.  (1987).  Masking  fields:  A  massively  parallel  architecture 
for  learning,  recognizing,  and  predicting  multiple  groupings  of  patterned  data.  In  M. 
Caudill  and  C.  Butler  (Eds.),  Proceedings  of  the  IEEE  international  conference  on  neural 
networks ,  II,  787-794.  (+*) 

13.  Cohen,  M.A.,  Grossberg,  S.,  and  Stork,  D.G.  (1987).  Recent  developments  in  a  neural 
model  of  real-time  speech  analysis  and  synthesis.  In  M.  Caudill  and  C.  Butler  (Eds.), 
Proceedings  of  the  IEEE  international  conference  on  neural  networks,  TV,  443-454.  (+) 

14.  Cohen,  M.A.,  Grossberg,  S.,  and  Stork,  D.G.  (1988).  Speech  perception  and  production 
by  a  self-organizing  neural  network.  In  Y.C.  Lee  (Ed.),  Evolution,  learning,  cognition, 
and  advanced  architectures.  Hong  Kong:  World  Scientific  Publishers.  (+) 

15.  Grossberg,  S.  11987).  Content-addressable  memory  storage  by  neural  networks:  A 
general  model  and  global  Liapunov  method.  In  E.L.  Schwartz  (Ed.),  Computational 
neuroscience.  Cambridge,  MA:  MIT  Press.  (+) 

16.  Grossberg,  S.  (1988).  Nonlinear  neural  networks:  Principles,  mechanisms,  and  architec¬ 
tures.  Neural  Networks,  1,  17-61.  (+) 

17.  Grossberg,  S.  (1988).  Preattentive  visual  perception:  Emergent  segmentation  and  feat- 
ural  filling-in.  In  L.  Spillmann  and  J.S.  Werner  (Eds.),  The  neurophysiological  foun¬ 
dations  of  visual  perception.  New  York:  Academic  Press. 

18.  Grossberg,  S.  (Ed.)  (1988).  Neural  networks  and  natural  intelligence.  Cambridge, 
MA:  MIT  Press'  (+*) 

19.  Grossberg,  S.  and  Gutowski,  W.  (1987).  Neural  dynamics  of  decision  making  under  risk: 
Affective  balance  theory.  In  M.  Caudill  and  C.  Butler  (Eds.),  Proceedings  of  the  IEEE 
international  conference  on  neural  networks,  HI,  31-38.  (+) 

20.  Grossberg,  S.  and  Gutowski,  W.  (1987).  Neural  dynamics  of  decision  making  under 
risk:  Affective  balance  and  cognitive-emotional  interactions.  Psychological  Review,  94, 
300-318.  (+) 

21.  Grossberg,  S.  and  Levine,  D.  (1987).  Neural  dynamics  of  attentionally  modulated 
Pavlovian  conditioning:  Blocking,  interstimulus  interval,  and  secondary  reinforcement. 
Applied  Optics,  26,  5015-5030.  (+*) 

22.  Grossberg,  S.,  Levine,  D.,  and  Schmajuk,  N.A.  (1988).  Predictive  regulation  of  associa¬ 
tive  learning  in  a  neural  network  by  reinforcement  ana  attentive  feedback.  International 
Journal  of  Neurology,  special  issue  on  Memory,  in  press.  (+) 

23.  Grossberg,  S.  and  Marshall,  J.  (1987).  A  computational  model  of  how  cortical  complex 
cells  multiplex  information  about  position,  contrast,  orientation,  spatial  frequency,  and 
disparity.  In  M.  Caudill  and  C.  Butler  (Eds.),  Proceedings  of  the  IEEE  international 
conference  on  neural  networks,  TV,  203-214.  (*) 

24.  Grossberg,  S.  and  Marshall,  J.  (1989).  Stereo  boundary  fusion  by  cortical  complex  cells: 
A  system  of  maps,  filters,  and  feeuback  networks  for  multiplexing  distributed  data.  Neural 
Networks,  in  press.  (+*) 

25.  Grossberg,  S.  and  Mingoila,  E.  (1987).  A  neural  network  architecture  for  preattentive 
vision:  Multiple  scale  segmentation  and  regularization.  In  M.  Caudill  and  C.  Butler  (Eds.), 
Proceedings  of  the  IEEE  international  conference  on  neural  networks,  IV,  177-184.  (*) 

26.  Grossberg,  S.  and  Mingoila,  E.  (1987).  Neural  dynamics  of  surface  perception:  Bound¬ 
ary  webs,  illuminants,  and  shape-from-shading.  Computer  Vision,  Graphics,  and  Image 
Processing,  37,  116-165.  (+*) 


27.  Grossberg,  S.  and  Schmajuk,  N.A.  (1987).  A  neural  network  architecture  for  attentionally- 
modulated  Pavlovian  conditioning:  Conditioned  reinforcement,  inhibition,  and  opponent 
processing.  In  M.  Caudill  and  C.  Butler  (Eds.),  Proceedings  of  the  IEEE  international 
conference  on  neural  networks ,  II,  795-804.  (+*) 

28.  Grossberg,  S.  and  Schmajuk,  N.A.  (1987).  Neural  dynamics  of  attentionally  modulated 
Pavlovian  conditioning:  Conditioned  reinforcement,  inhibition,  and  opponent  processing. 
Psychobiology ,  15,  195-240.  (+*) 

29.  Grossberg,  S.  and  Schmajuk,  N.A.  (1989).  Neural  dynamics  of  adaptive  timing  and 
temporal  discrimination  during  associative  learning.  Neural  Networks ,  in  press.  (+) 

30.  Grossberg,  S.  and  Todorovic,  D.  (1987).  A  neural  network  architecture  for  brightness 
perception  under  constant  and  variable  illumination  conditions.  In  M.  Caudill  and  C. 
Butler  (Eds.),  Proceedings  of  the  IEEE  international  conference  on  neural  networks ,  IV, 
185-192.  (*) 

31.  Grossberg,  S.  and  Todorovid,  D.  (1988).  Neural  dynamics  of  1-D  and  2-D  brightness 
perception:  A  unified  model  of  classical  and  recent  phenomena.  Perception  and  Psy¬ 
chophysics,  43,  241-277.  (*) 

32.  Grossberg,  S.  and  Todorovid,  D.  (1989).  Solving  the  brightness-from-luminanceproblem: 
A  neural  architecture  for  invariant  brightness  perception.  In  S.J.  Hanson  and  C.  Olson 
(Eds.),  Connectionist  modeling  and  brain  function:  The  developing  interface. 
Cambridge,  MA:  MIT  Press.  (+*) 

4-  Also  supported  in  part  by  the  National  Science  Foundation. 

*  Also  supported  in  part  by  the  Army  Research  Office. 


URI  ANNUAL  REPORT 
Year  2 

Gail  A.  Carpenter 
Northeastern  University 
and 

Center  for  Adaptive  Systems 
PUBLICATIONS 

1.  Carpenter,  G.A.  and  Grossberg,  S.  (1988).  The  ART  of  adaptive  pattern  recognition  by 
a  self-organizing  neural  network.  Computer  (special  issue  on  Artificial  Neural  Systems), 
21,  77-88. 

2.  Carpenter,  G.A.  and  Grossberg,  S.  (1989).  Search  mechanisms  for  hierarchical  Adaptive 
Resonance  Theory  (ART)  architectures.  In  preparation. 

3.  Carpenter,  G.A.,  Grossberg,  S.,  and  Mehanian,  C.  (1989).  Invariant  recognition  of 
cluttered  scenes  by  a  self-organizing  ART  architecture,  I:  CORT-X  boundary  segmentation. 
Neural  Networks,  in  press. 

4.  Carpenter,  G.A.  (1989).  Pattern  recognition,  associative  learning,  and  self-organization. 
Review  article  based  on  tutorial  lecture  at  First  Annual  INNS  Meeting,  Neural  Networks, 
in  preparation. 


This  portion  of  the  URI  contract  has  led  to  major  progress  on  Modeling  Project  6H 
(p.47): 

H.  Modeling  Project:  Embedding  the  Coding  Module  in  a  Network  Hierarchy 

The  computational  principles  which  enable  an  ART  circuit  to  do  category  learning  are 
universal:  they  are  not  restricted  to  particular  problem  domains,  such  as  speech  or  vision. 
For  example,  a  key  design  constraint  is  that  a  pattern  should  be  separable,  under  appropri¬ 
ate  vigilance  conditions,  from  another  pattern  with  which  it  may  share  many  features  (e.g., 
nested  inputs).  Once  the  formal  problems  listed  herein  have  been  solved,  ART  networks 
can  be  used  to  solve  many  specialized  perceptual  and  cognitive  domains.  Indeed,  they 
were  originally  derived  from  large  data  bases  within  such  data  domains. .  In  such  appli¬ 
cations,  different  preprocessing  stages  can  serve  as  precursors  of  the  ART  input  patterns. 
Neural  circuits  which  generate  preattentively  completed  segmentations  of  a  visual  image 
before  these  completed  segmentations  generate  inputs  to  an  ART  network  have  recently 
been  constructed  (see  Section  1).  In  applications  to  adaptive  speech  recognition,  inputs 
are  encoded  as  STM  patterns  of  temporal  order  information  across  item  representations 
before  these  STM  patterns  generate  inputs  to  an  ART  network  (Appendix  3).  Thus,  in 
this  stage  of  the  work,  we  will  plug  specialized  ART  circuits  into  network  hierarchies  to 
test  their  ability  to  discover,  learn,  and  recognize  invariant  properties  of  important  classes 
of  vision,  speech,  and  cognitive  codes. 


Each  of  the  four  articles  listed  above  has  contributed  to  this  project,  as  follows. 
Article  1  begins  with  a  review  of  the  ART  1  and  ART  2  neural  network  modules  that 
carry  out  stable  self-organization  of  recognition  codes.  The  article  goes  on  to  describe 


how  an  ART  module  can  be  embedded  in  neural  network  hierarchies  for  a  variety  of  tasks. 
These  include: 

(A)  invariant  visual  pattern  recognition,  with  the  ART  2  module  attached  to  a  pre¬ 
processor  that  regularizes  and  completes  boundaries  and  factors  out  translation,  rotation, 
and  scaling  invariants; 

(B)  a  neural  network  hierarchy  that  incorporates  reinforcement  and  recall,  as  well  as 
ART  recognition;  and 

(C)  a  description  of  a  cascade  of  ART  systems  embedded  in  a  hierarchy  to  model  the 
perception  and  production  of  speech. 

Article  2  solves  some  key  technical  difficulties  that  had  stood  in  the  way  of  moving 
from  the  minimal  ART  1  and  ART  2  designs  to  more  sophisticated  network  hierarchies.  In 
particular,  a  new  search  mechanism  allows  the  ART  2  systems  to  be  naturally  embedded 
into  network  hierarchies  while  retaining  the  basic  design  of  the  minimal  modules.  In 
particular,  the  new  search  mechanism  works  well  in  systems  with  either  fast  or  slow  learning 
and  with  either  compressed  or  distributed  codes.  This  new  search  mechanism,  as  well  as 
the  general  problem  of  designing  ART  systems  for  neural  network  hierarchies,  is  the  subject 
of  ongoing  research. 

Article  3  is  the  first  in  a  series  on  the  invariant  visual  pattern  recognition  problem 
mentioned  above.  Article  3  describes  a  new  filter  for  the  preattentive  regularization  and 
completion  of  image  boundaries.  The  CORT-X  filter  uses  multiple  scales  to  carry  out  many 
of  the  computations  of  the  Gross berg-Mingolla  Boundary  Contour  System  (BCS),  but  the 
feedforward  CORT-X  filter  can  allow  more  rapid  computation  than  does  the  feedback  BCS 
system,  albeit  with  some  loss  of  pattern  processing  capabilities. 

Article  4  is  primarily  a  review  article  describing  the  history  of  neural  networks  for  pat¬ 
tern  recognition,  associative  learning,  and  self-organization.  However,  being  organized  by 
theme  rather  than  chronology,  the  article  leads  naturally  to  a  new  system  that  incorporates 
two  or  more  ART  2  modules  into  a  network  hierarchy  for  associative  learning. 


URI  ANNUAL  REPORT 
Year  2 

Ennio  Mingolla 
Center  for  Adaptive  Systems 

Modeling  Project:  3-D  Shape  from  2-D  Images  of  Surfaces 

Software  has  been  developed  on  a  Silicon  Graphics  IRIS  3130  workstation  to  rapidly 
generate  parametrically  varied  families  of  srufaces  illuminated  by  single  or  multiple  light 
sources.  New  graduate  students  in  the  Cognitive  and  Neural  Systems  Program  are  learning 
about  the  BCS  in  anticipation  of  carrying  out  simulations  of  multiple-scale  boundary  webs. 


Experiments  on  Surface  Perception 

Since  the  arrival  and  installation  of  the  Silicon  Graphics  IRIS  3130  workstation  in 
June,  1987,  Dr.  Mingolla  has  developed  number  of  software  tools  for  manipulating  surface 
reflectance  characteristics  of  simulated  surafces.  The  shading  model  of  equation  (1)  (page 
26)  has  been  implemented,  along  with  some  special  techniques  for  introducing  smooth 
nonlinear  distortions  of  certain  image  pixel  intensities.  As  a  result,  surface  appearance 
can  be  controlled  either  by  conventional  computer  graphics  techniques  (the  shading  model 
equation)  or  by  manipulations  having  no  analog  in  the  natural  image  generation  process. 
The  latter  manipulations,  however,  enable  testing  of  theoretical  ideas.  Thus,  following 
implications/predictions  of  the  theory  of  Grossberg  and  his  colleagues,  he  was  able  to 
construct  a  display  in  which  one  half  of  an  extended  surface  could  be  made  to  appear 
either  glossy  or  dull  without  any  change  of  its  pixel  intensities.  Instead,  the  context  of 
adjacent  pixel  intensities  was  manipulated  by  nonlinear  distortions  in  such  a  way  that  a 
certain  portion  of  the  unchanged  surface  would  in  one  case  be  processed  as  structural  (i.e., 
part  of  a  boundary  webl  and  in  another  case  as  featural  (i.e.,  part  of  FCS  spreading).  In 
the  former  case  the  unchanged  region  appeared  dull,  in  the  latter  case  shiny. 

Preliminary  results  from  this  research  were  reported  at  the  Fourth  International  Con¬ 
ference  on  Event  Perception,  Trieste,  Italy,  August  1987,  and  additional  work  was  presented 
at  a  topical  meeting  on  Shape-from-Shading  in  the  fall  of  1988  in  Toronto,  Canada.  A  more 
sophisticated  version  of  the  nonlinear  intensity  manipulation  scheme  is  being  developed. 
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The  proposed  work  concerned  the  mechanisms  underlying  perception  of 
color. 

I  Mondrian  displays. 

We  have  completed  five  studies  of  color  constancy  in  Land-type 
'Mondrian'  displays;  that  is,  displays  of  several  abutted  homogeneous 
colored  'papers',  which  in  our  work  have  been  simulated  on  a  high- 
resolution  color  monitor.  This  work  has  been  done  with  L.  E.  Arend  of 
the  Eye  Research  Institute,  as  part  of  the  URI  grant.  Observers  are 
required  to  make  matches  between  standard  and  comparison  displays  lit 
(in  the  simulation)  by  different  phases  of  daylight.  We  ask  whether 
color  constancy  can  be  obtained  (i.e.,  whether  the  matches  would  be 
independent  of  illuminant.)  Our  studies  feature  (1)  control  of 
adaptational  state;  (2)  control  of  the  observer's  task  (to  match 
surface  appearances,  or  to  match  chromaticities) ;  (3)  control  of 
stimulus  complexity  (annular  versus  Mondrian-type  displays) ;  (4)  use 
of  standard  and  comparison  stimuli  with  the  same  surrounds,  to  avoid 
unequal  induction;  (5)  use  of  binocular  vision,  to  avoid  the  standard 
haploscopic  method  in  which  standard  and  comparison  are  presented  to 
different  eyes  and  so  fall  on  differently-adapted  retinae. 

Results  of  Study  1  (supported  by  previous  grants)  show  that  when 
the  state  of  adaptation  is  kept  roughly  constant  by  requiring  the 
observer  to  scan  back  and  forth  between  comparison  and  standard 
displays,  the  extent  of  color  constancy  depends  primarily  on 
instruction  and  only  very  secondarily  on  stimulus  complexity  (Arend 
and  Reeves,  1986) .  Color  constancy  is  strong  with  surface  appearance 
('paper')  matches,  but  weak  or  absent  in  chromaticity  ('hue  and 
saturation')  matches.  This  important  result  is  contrary  to  many 
established  models  for  color  constancy  (eg,  Retinex) . 

New  work,  done  under  the  URI  grant  (first  and  second  years)  and 
partly  reported  at  ARVO  (Arend,  et.  al.,  1987;  1988)  shows  that  the 
same  surprising  pattern  of  results  is  obtained  when  luminance 
variations  of  up  to  19:1  are  added  to  the  displays  (Study  2;  near 
equi- luminance  displays  were  used  in  Study  1) . 

We  also  find  this  pattern  of  results  when  standard  and  comparison 
displays  are  alternated  (Study  3),  rather  than  being  presented 
simultaneously  as  in  Study  1.  This  is  important  because  color 
constancy  would  not  be  expected  across  two  simultaneous  displays  if 
the  visual  system  extracted  a  mean  illuminant  over  the  entire  visual 
field.  Therefore,  the  absence  of  color  constancy  in  the  chromaticity 
task  even  in  this  experiment  is  particularly  significant. 

In  Study  4,  we  have  found  color  constancy  for  the  chromaticity 
instruction,  when  adaptation  is  appropriately  varied  by  interposing 
homogeneous  adapting  fields  between  stimulus  presentations.  This 
result  would  be  expected  from  Von  Kries  -type  considerations  and 
serves  as  a  control  on  our  methods. 

In  Study  5,  we  asked  subjects  to  set  a  single  target  paper  in  a 
single  Mondrian  to  a  specified  unique  color  (eg,  yellow) .  Once  more, 
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we  have  found  a  large  effect  of  instructions:  color  constancy  is  good 
in  the  paper  task,  but  not  in  the  chromaticity  task.  This  result  is 
important,  because  one  might  have  argued  that  a  form  of  temporal 
integration  (as  well  as  spatial)  across  displays  occurred  in  Study  3, 
once  gain  preventing  color  constancy  even  though  the  displays  in  Study 
3  were  alternated.  Study  5  controlled  for  this  because  only  one 
display  is  seen  in  each  block  of  trials. 

We  have  developed  an  index  for  color  constancy  in  order  to  quantify 
its  extent.  On  (u' ,  v')  CIE  co-ordinates,  which  are  approximately 
equally-spaced  in  jnd.s,  we  let  c=  the  distance  from  the  test  co¬ 
ordinate  to  the  match  coordinate  divided  by  the  distance  that  would  be 
found  if  color  constancy  were  perfect.  We  then  average  the  values  of  c 
found  for  the  five  different  test  chromaticj ties  that  we  use  (red, 
green,  yellow,  grey,  and  blue).  Mean  c  equals  1.0  if  color  constancy 
is  perfect,  and  =0  if  subjects  make  chromaticity  matches  (and  thus 
show  no  color  constancy).  Over  all  conditions  and  subjects,  we  find 
mean  c  =  0.6  for  surface  matches  and  for  chromaticity  matches.  These 
values  for  c  vary  little  over  experiments,  though  they  do  vary  over 
subjects;  our  ' worst '  subject  in  the  paper  match  averaged  c=0.4  (but 
he  also  obtained  c=.2  in  the  chromaticity  matches). 

II.  Adaptational  history. 

We  have  tested,  calibrated,  and  programmed  our  Generation  V  SRI 
Purkinje  Eye-tracker,  in  order  to  study  the  effect  of  the  adaptational 
history  of  an  observer  whose  eyes  are  free  to  scan  the  display 
naturally.  We  proposed  a  series  of  experiments  based  on  this 
technology  in  the  original  URI  grant.  However,  the  work  has  gone  very 
slowly,  partly  due  to  technical  problems  with  the  Eye  tracker,  which 
have  been  fixed.  As  yet  we  have  no  publishable  data,  but  we  have  made 
useable  recordings.  We  plan  to  perform  the  proposed  experiments  in  the 
third  year  of  the  grant. 

III.  Depth,  lightness,  and  brightness. 

In  separate  work  with  achromatic  displays  which  simulate 
experiments  first  done  by  Alan  Gilchrist  (1977),  we  have  found  that 
perceived  depth  influences  the  lightness,  but  not  the  brightness,  of  a 
target.  (Gilchrist  did  not  make  this  distinction  operationally.) 

The  target  is  surrounded  by  surfaces  which,  due  to  interposition 
and/or  stereo  cues,  appear  in  different  depth  planes  (Schirillo,  1987; 
Reeves  and  Schirillo,  1987) .  Subjects  match  either  the  lightness  or 
the  brightness  of  a  target  area  to  a  Munsell  scale  presented  below  the 
screen.  This  work  has  been  done  with  moderate,  large,  and  very  large 
(900:1)  contrast  ratios  (the  latter  like  Gilchrist's).  We  have  varied 
the  background  of  the  Munsell  scale  and  the  method  of  matching 
(multiple  choice  or  adjustment) .  In  each  case,  depth  has  no  effect  on 
brightness,  but  changes  the  Munsell  value  by  up  to  3  points.  The 
reliability  of  this  finding  suggests  that  brightness  is  computed 
before,  and  lightness  after,  the  construction  of  a  3-D  representation 
of  the  image.  This  work  was  completed  in  the  second  year  of  the  URI 
grant  and  is  being  written  up  for  publication. 

These  studies  have  been  conducted  with  Lawrence  Arend  of  the  Eye 
Research  Institute,  and  James  Schirillo,  who  is  a  PhD  student  at 
Northeastern,  both  supported  by  the  URI  grant. 


IV.  Transient  tritanopia 

During  the  second  year  of  the  grant  period,  work  has  been  done  with 
Michael  Rudd  and  Steven  Grossberg  (Center  for  Adaptive  Systems)  to 
model  earlier  work  on  transient  tritanopia  and  its  analogues  in  the 
red/green  system.  The  essential  finding  is  that  turning  off  or  down  a 
colored  background  can  greatly  reduced  the  sensitivity  of  the 
red/green  and  yellow/blue  hue  pathways,  even  though  normal  dark 
adaptation  (recovery  of  sensitivity)  occurs  in  the  luminance  channel. 
The  current  best  model  for  this,  by  Pugh  and  Mollon,  postulates  a  slow 
(15  sec  time-constant)  'restoring  force'  which  bu'.lds  up  during  light 
adaptation  and  suddenly  rebounds  when  the  light  is  turned  off.  This 
model,  however,  cannot  handle  the  result  (of  Reeves)  that  transient 
tritanopia  can  be  completely  eliminated  by  1  Hz  flicker  during  the 
adaptation  period.  The  new  model  (Reeves,  1988)  employs  Grossberg's 
theoretical  circuits  which  can  predict  the  relevant  phenomena 
qualitatively.  We  are  currently  refining  the  model  and  attempting  to 
broaden  the  logical  and  empirical  support  for  it. 
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Two  ideas  are  explored,  both  of  which  concern  the  organization  of  cone  classes  in  the  subsequent 
connections  of  the  visual  system. 

1:  The  simplest  notion  of  isoluminance  is  problematic.  If  isoluminance  is  determined  by  a 
weighted  sum  of  the  input  from  individual  cone  classes  then  all  the  phenomena  reported  to  occur 
with  isoluminant  stimuli  should  be  present  whatever  the  spectral  composition  of  the  colors  used. 
Thus,  for  example,  the  disappearance  of  the  kinetic  depth  effect  should  occur  equally  well  using  green 
dots  on  a  grey  background  or  green  dots  on  a  red  background.  Our  work  demonstrates  that  this  is  not 
the  case.  Using  both  subjective  and  objective  criteria,  the  quality  of  isoluminance  matches  (and  the 
psychophysical  phenomena  associated  with  isoluminant  stimuli)  vary  dramatically  with  the  spectral 
composition  of  the  colored  stimuli.  The  subjective  measure  is  simply  the  reported  quality  of  match. 
The  objective  measure  is  the  variability  of  isoluminance  settings.  In  particular,  when  the  colors  are  a 
grey  and  green,  the  accuracy  (repeatability)  of  isolumance  matches  is  extremely  good — about  1%  (as 
good  as  making  a  luminance  match  with  a  single  hue).  And  subjects  experience  the  match  as  being 
excellent — in  the  case  of  the  kinetic  depth  effect,  all  sense  of  depth  is  lost  at  isoluminance.  On  the 
other  hand,  when  the  colors  are  a  saturated  red  and  saturated  blue,  the  variability  of  isoluminance 
settings  is  much  higher — about  15%.  The  subjective  phenomena  are  much  weaker  as  well.  Subjects 
continue  to  experience  such  phenomena  as  the  kinetic  depth  effect  at  all  relative  settings  of  the  two 
colors  (in  contrast  to  some  reports).  This  finding  is  robust  across  a  number  of  criteria — the  kinetic 
depth  effect,  stereopsis,  hetereochromatic  brightness  matches,  and  flicker. 

2:  It  is  possible  (using  computer  generated  stimuli)  to  present  a  broad  range  of  traditional  psy¬ 
chophysical  stimuli  so  that  only  a  single  class  of  cones  can  be  used  in  the  perception  of  those  stimuli. 
This  is  an  extension  of  the  body  of  work  done  using  the  “cardinal”  directions  of  color  space,  to  use 
the  term  coined  by  Krauskopf,  et  al.  Stimuli  are  designed  so  that  the  spatial  and  temporal  pattern  on 
(for  instance)  two  of  the  three  cone  classes  is  uniform.  The  third  cone  class  is  presented  with  spatial 
and  temporal  patterns  of  arbitrary  complexity.  The  simplest  experiment  is  to  measure  the  contrast 
threshold  for  detection  of  a  spot  by  a  single  cone  class.  The  results  (which  replicate  the  work  of  sev¬ 
eral  other  laboratories)  reveal  the  cardinal  directions  to  be  one  luminance-like  channel  and  two  color 
difference  channels.  More  general  stimuli  are  possible.  For  instance,  one  can  present  a  random  dot 
stereogram  so  that  the  pattern  on  the  left  eye  is  seen  only  by  the  long-wave  cones  while  the  pattern  on 
the  right  eye  is  seen  only  by  the  middle  wave  cones.  Stereopsis  is  still  obtained  (and  can  be  quantita¬ 
tively  assessed,  in  terms  of  the  contrast  needed  to  be  perceived).  More  than  one  intensity  level  can  be 
presented  at  once,  so  that  Mach  Band  patterns,  Simultaneous  Contrast  patterns,  and  even  Mondrian- 
like  patterns  can  be  presented  to  isolated  cone  classes.  The  only  restriction  is  that  the  maximum 
contrast  cannot  be  too  large — about  15%  on  the  long  and  middle  wave  cone  classes  and  about  80% 
on  the  short  wave  cones.  This  limitation  is  imposed  by  the  overlap  of  the  cone  pigment  absorptions. 
These  techniques  for  isolating  the  activities  of  cone  classes  have  already  been  used  in  some  neuro¬ 
physiological  work  and  promise  to  be  a  useful  tool  in  determining  connections  in  the  visual  system. 
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The  research  in  the  present  article  was  designed  to  investigate  the  phenomenon  of 
stereoscopic  transparency,  in  which  overlapping  surfaces  are  perceived  simultaneously  at 
different  depths  in  the  same  visual  direction.  Four  experiments  are  reported  that  examined 
observers’  abilities  to  achieve  this  phenomenon  over  a  wide  range  of  stimulus  conditions. 

The  results  indicate  that  (1)  the  perceptual  segregation  of  overlapping  transparent  surfaces  I 

is  significantly  more  difficult  that  are  comparable  judgments  for  opaque  surfaces;  (2)  the 
perception  of  transparency  is  impaired  by  increased  depth  differences  between  the  over¬ 
lapping  surfaces  or  by  increased  element  density;  and  (3)  the  effect  is  facilitated  when  the 
overlapping  depth  planes  are  distinguished  by  color,  but  not  when  they  are  distinguished 
by  element  orientation.  The  theoretical  significance  of  these  results  is  considered  within 
the  context  of  current  competitive/cooperative  models  of  stereo  matching.  j 
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Perceptual  depth  inversion  of  smoothly  curved  surfaces  was  investigated  in  two  exper¬ 
iments.  Observers  made  relative  depth  judgments  for  pairs  of  points  on  three  computer 
generated  surfaces  that  were  presented  in  two  orientations — upright  and  upside  down.  The 
same  three  surfaces  were  defined  by  shading  in  the  first  experiment,  and  by  contours  in 
the  second  experiment.  The  results  of  the  experiments  indicate  that:  (a)  For  both  shaded 
and  contoured  depictions  of  smoothly  curved  surfaces,  a  180-degree  rotation  in  the  picture 
plane  causes  perceptual  reversals  such  that  some  regions  of  the  undulating  surfaces  appear 
to  reverse  in  depth,  (b)  Smoothly  curved  surfaces  do  not  reverse  as  wholes.  Rather,  they 
undergo  a  piecemeal  reversal,  where  certain  local  regions  reverse  in  depth,  and  others  re¬ 
main  stable,  (c)  Some  surfaces  are  more  susceptible  to  perceptual  reversal  than  others, 
(d)  The  presence  of  occlusion  contours  decreases  the  likelihood  of  perceptual  reversals. 
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The  research  described  in  the  present  article  was  designed  to  investigate  how  patterns 
of  optical  texture  provide  information  about  the  three-dimensional  structure  of  objects 
in  space.  Four  experiments  were  performed  in  which  observers  were  asked  to  judge  the 
perceived  depth  of  simulated  ellipsoid  surfaces  under  a  variety  of  experimental  conditions. 
The  results  revealed  that  (a)  judged  depth  increases  linearly  with  simulated  depth  although 
the  slope  of  this  relation  varies  significantly  among  different  types  of  texture  patterns, 
(b)  Random  variations  in  the  sizes  and  shapes  of  individual  surface  elements  have  no 
detectable  effect  on  observers’  judgments,  (c)  The  perception  of  three-dimensional  form  is 
quite  strong  for  surfaces  displayed  under  parallel  projection,  but  the  amount  of  apparent 
depth  is  slightly  less  than  for  identical  surfaces  displayed  under  polar  projection,  (d) 
Finally,  the  perceived  depth  of  a  surface  is  eliminated  if  the  optical  elements  in  a  display 
jure  not  sufficiently  elongated  or  if  they  are  not  approximately  aligned  with  one  another.  A 
theoretical  explanation  of  these  findings  is  proposed  based  on  the  neural  network  analysis 
of  Grossberg  and  Mingolla. 
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Several  experiments  were  performed  in  an  effort  to  determine  the  different  combinations 
of  display  parameters  that  can  reliably  elicit  a  perceptually  compelling  impression  of  a 
solid  object  rotating  rigidly  in  three-dimensional  space.  All  of  the  displays  were  computer¬ 
generated  simulations  of  rigid  objects  rotating  in  depth  under  parallel  projection.  The 
parameters  investigated  included  the  number  of  frames  in  the  apparent  motion  sequence, 
the  temporal  and  spatial  displacements  between  frames,  the  number  of  elements  in  the 
depicted  object,  and  the  structural  organization  of  those  elements.  The  results  indicated 
that  there  are  large  interactions  among  these  different  parameters  in  their  effects  on  the 
perception  of  structure  from  motion. 
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In  theoretical  analyses  of  visual  form  perception,  it  is  often  assumed  that  the  3- 
dimensional  structures  of  smoothly  curved  surfaces  are  perceptually  represented  as  point- 
by-point  mappings  of  metric  depth  and/or  orientation  relative  to  the  observer.  This  paper 
describes  an  alternative  theory,  in  which  it  is  argued  that  our  visual  knowledge  of  smoothly 
curved  surfaces  is  often  restricted  to  local,  nonmetric  order  relations.  A  fundamental  pre¬ 
diction  of  this  analysis  is  that  relative  depth  judgments  between  any  two  surface  regions 
should  be  dramatically  influenced  by  the  monotonicity  of  depth  change  (or  lack  of  it)  along 
the  intervening  portions  of  the  surface  through  which  they  are  separated.  This  prediction  is 
confirmed  in  a  series  of  experiments  using  surfaces  depicted  with  either  shading  or  texture. 
Additional  experiments  are  reported,  moreover,  which  demonstrate  that  smooth  occlusion 
contours  are  a  primary  source  of  information  about  the  ordinal  structure  of  a  surface,  and 
that  the  depth  extrema  in  between  contours  can  be  optically  specified  by  differences  in 
luminance  at  the  points  of  occlusion. 
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PROBING  COGNITIVE  PROCESSES  THROUGH  THE 
STRUCTURE  OF  EVENT-RELATED  POTENTIALS  DURING  LEARNING: 
AN  EXPERIMENTAL  AND  THEORETICAL  ANALYSIS 


Jean-Paul  Banquetf  and  Stephen  Grossbergt 
Applied  Optics,  1987,  26,  4931—4946 
ABSTRACT 

Data  reporting  correlated  changes,  due  to  learning,  in  the  amplitudes  and  chronome- 
try  of  several  event  related  potentials  (ERPs)  are  compared  with  neural  explanations  and 
predictions  of  the  adaptive  resonance  theory.  The  ERP  components  processing  negativity 
(PN),  early  positive  wave  (P120),  N200,  and  P300  covary  with  model  processes  of  atten- 
tional  priming  and  top-down  expectancy  learning,  matching  of  bottom-up  input  patterns 
with  learned  top-down  expectations,  mismatch- mediated  activation  of  the  orienting  sub¬ 
system,  reset  by  the  orienting  subsystem  of  recognition  codes  in  short  term  memory,  and 
direct  activation  of  recognition  codes  via  a  bottom-up  adaptive  filter.  These  model  mech¬ 
anisms  enable  a  recognition  code  to  be  learned  in  a  self-stabilizing  fashion  in  response  to 
an  input  environment  of  arbitrary  complexity.  Thus  spatiotemporal  correlations  among 
several  ERPs  during  learning  provide  important  evidence  in  support  of  postulated  neural 
mechanisms  for  self-stabilizing  self-organization  of  cognitive  recognition  codes. 
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NEURAL  DYNAMICS  OF  PLANNED  ARM  MOVEMENTS: 
EMERGENT  INVARIANTS  AND  SPEED-ACCURACY  PROPERTIES 
DURING  TRAJECTORY  FORMATION 

Daniel  Bullockf  and  Stephen  Grossbergf 
Psychological  Review,  1988,  95,  49-90 
ABSTRACT 

A  real-time  neural  network  model,  called  the  Vector  Integration  to  Endpoint,  or  VITE, 
Model,  is  developed  and  used  to  quantitatively  simulate  behavioral  and  neural  data  about 
planned  and  passive  arm  movements.  Invariants  of  arm  movements  emerge  through  net¬ 
work  interactions  rather  than  through  an  explicitly  precomputed  trajectory.  Motor  plan¬ 
ning  occurs  in  the  form  of  a  Target  Position  Command,  or  TPC,  which  specifies  where 
the  arm  intends  to  move,  and  an  independently  controlled  GO  command,  which  specifies 
the  movement’s  overall  speed.  Automatic  processes  convert  this  information  into  an  arm 
trajectory  with  invariant  properties.  These  automatic  processes  include  computation  of  a 
Present  Position  Command,  or  PPC,  and  a  Difference  Vector,  or  DV.  The  DV  is  the  differ¬ 
ence  of  the  PPC  and  the  TPC  at  any  time.  The  PPC  is  gradually  updated  by  integrating 
the  DV  through  time.  The  GO  signal  multiplies  the  DV  before  it  is  integrated  by  the  PPC. 
The  PPC  generates  an  outflow  movement  command  to  its  target  muscle  groups.  Opponent 
interactions  regulate  the  PPC’s  to  agonist  and  antagonist  muscle  groups.  This  system  gen¬ 
erates  synchronous  movements  across  synergetic  muscles  by  automatically  compensating 
for  the  different  total  contractions  that  each  muscle  group  must  undergo.  Quantitative  sim¬ 
ulations  are  provided  of  Woodworth’s  Law,  of  the  speed-accuracy  trade-off  known  as  Fitts’ 
Law,  of  isotonic  arm  movement  properties  before  and  after  deafferentation,  of  synchronous 
and  compensatory  “central  error  correction”  properties  of  isometric  contractions,  of  veloc¬ 
ity  amplification  during  target  switching,  of  velocity  profile  invariance  and  asymmetry,  of 
the  changes  in  velocity  profile  asymmetry  at  higher  movement  speeds,  of  the  automatic 
compensation  for  staggered  onset  times  of  synergetic  muscles,  of  vector  cell  properties  in 
precentral  motor  cortex,  of  the  inverse  relationship  between  movement  duration  and  peak 
velocity,  and  of  peak  acceleration  as  a  function  of  movement  amplitude  and  duration.  It 
is  shown  that  TPC,  PPC,  and  DV  computations  are  needed  to  actively  modulate,  or  gate, 
the  learning  of  associative  maps  between  TPC’s  of  different  modalities,  such  as  between 
the  eye-head  system  and  the  hand-arm  system.  By  using  such  an  associative  map,  looking 
at  an  object  can  activate  a  TPC  of  the  hand-arm  system,  as  Piaget  noted.  Then  a  VITE 
circuit  can  translate  this  TPC  into  an  invariant  movement  trajectory.  An  auxiliary  circuit, 
called  the  Passive  Update  of  Position,  or  PUP,  Model,  is  described  for  using  inflow  signals 
to  update  the  PPC  during  passive  arm  movements  due  to  external  forces.  Other  uses  of 
outflow  and  inflow  signals  are  also  noted,  such  as  for  adaptive  linearization  of  a  nonlin¬ 
ear  muscle  plant,  and  sequential  read-out  of  TPC’s  during  a  serial  plan,  as  in  reaching 
and  grasping.  Comparisons  are  made  with  other  models  of  motor  control,  such  as  the 
mass-spring  and  minimum-jerk  models. 
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ART  2:  SELF-ORGANIZATION  OF  STABLE  CATEGORY 
RECOGNITION  CODES  FOR  ANALOG  INPUT  PATTERNS 
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ABSTRACT 

Adaptive  resonance  architectures  are  neural  networks  that  self-organize  stable  pattern 
recognition  codes  in  real-time  in  response  to  arbitrary  sequences  of  input  patterns.  This 
article  introduces  ART  2,  a  class  of  adaptive  resonance  architectures  which  rapidly  self- 
organize  pattern  recognition  categories  in  response  to  arbitrary  sequences  of  either  analog 
or  binary  input  patterns.  In  order  to  cope  with  arbitrary  sequences  of  analog  input  pat¬ 
terns,  ART  2  architectures  embody  solutions  to  a  number  of  design  principles,  such  as  the 
stability-plasticity  tradeoff,  the  search-direct  access  tradeoff,  and  the  match-reset  tradeoff. 
In  these  architectures,  top-down  learned  expectation  and  matching  mechanisms  are  crit¬ 
ical  in  self-stabilizing  the  code  learning  process.  A  parallel  search  scheme  updates  itself 
adaptively  as  the  learning  process  unfolds,  and  realizes  a  form  of  real-time  hypothesis  dis¬ 
covery,  testing,  learning,  and  recognition.  After  learning  self-stabilizes,  the  search  process 
is  automatically  disengaged.  Thereafter  input  patterns  directly  access  their  recognition 
codes  without  any  search.  Thus  recognition  time  for  familiar  inputs  does  not  increase  with 
the  complexity  of  the  learned  code.  A  novel  input  pattern  can  directly  access  a  category 
if  it  shares  invariant  properties  with  the  set  of  familiar  exemplars  of  that  category.  A 
parameter  called  the  attentional  vigilance  parameter  determines  how  fine  the  categories 
will  be.  If  vigilance  increases  (decreases)  due  to  environmental  feedback,  then  the  system 
automatically  searches  for  and  learns  finer  (coarser)  recognition  categories.  Gain  control 
parameters  enable  the  architecture  to  suppress  noise  up  to  a  prescribed  level.  The  archi¬ 
tecture’s  global  design  enables  it  to  learn  effectively  despite  the  high  degree  of  nonlinearity 
of  such  mechanisms. 


t  Supported  in  part  by  the  Air  Force  Office  of  Scientific  Research  (AFOSR  F 49620-86- 
C-0037  and  AFOSR  F49620-87-C-0018),  the  Army  Research  Office  (ARO  DAAG-29-85- 
K-0095),  and  the  National  Science  Foundation  (NSF  DMS-86-11959). 

X  Supported  in  part  by  the  Air  Force  Office  of  Scientific  Research  (AFOSR  F49620-86- 
C-0037  and  AFOSR  F49620-87-C-0018),  the  Army  Research  Office  (ARO  DAAG-29-85- 
K-0095),  and  the  National  Science  Foundation  (NSF  IRI-84-17756). 


A  MASSIVELY  PARALLEL  ARCHITECTURE 
FOR  A  SELF-ORGANIZING 
NEURAL  PATTERN  RECOGNITION  MACHINE 
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ABSTRACT 

A  neural  network  architecture  for  the  learning  of  recognition  categories  is  derived. 
Real-time  network  dynamics  are  completely  characterized  through  mathematical  analysis 
and  computer  simulations.  The  architecture  self-organizes  and  self-stabilizes  its  recogni¬ 
tion  codes  in  response  to  arbitrary  orderings  of  arbitrarily  many  and  arbitrarily  complex 
binary  input  patterns.  Top-down  attentional  and  matching  mechanisms  are  critical  in 
self-stabilizing  the  code  learning  process.  The  architecture  embodies  a  parallel  search 
scheme  which  updates  itself  adaptively  as  the  learning  process  unfolds.  After  learning 
self-stabilizes,  the  search  process  is  automatically  disengaged.  Thereafter  input  patterns 
directly  access  their  recognition  codes  without  any  search.  Thus  recognition  time  does  not 
grow  as  a  function  of  code  complexity.  A  novel  input  pattern  can  directly  access  a  cate¬ 
gory  if  it  shares  invariant  properties  with  the  set  of  familiar  exemplars  of  that  category. 
These  invariant  properties  emerge  in  the  form  of  learned  critical  feature  patterns,  or  pro¬ 
totypes.  The  architecture  possesses  a  context-sensitive  self-scaling  property  which  enables 
its  emergent  critical  feature  patterns  to  form.  They  detect  and  remember  statistically 
predictive  configurations  of  featural  elements  which  are  derived  from  the  set  of  all  input 
patterns  that  are  ever  experienced.  Four  types  of  attentional  process — priming,  gain  con¬ 
trol,  vigilance,  and  intermodal  competition — are  mechanistically  characterized.  Top-down 
priming  and  gain  control  are  needed  for  code  matching  and  self-stabilization.  Attentional 
vigilance  determines  how  fine  the  learned  categories  will  be.  If  vigilance  increases  due  to 
an  environmental  disconfirmation,  then  the  system  automatically  searches  for  and  learns 
finer  recognition  categories.  A  new  nonlinear  matching  law  (the  2/3  Rule)  and  new  nonlin¬ 
ear  associative  laws  (the  Weber  Law  Rule,  the  Associative  Decay  Rule,  and  the  Template 
Learning  Rule)  are  needed  to  achieve  these  properties.  All  the  rules  describe  emergent 
properties  of  parallel  network  interactions.  The  architecture  circumvents  the  noise,  satu¬ 
ration,  capacity,  orthogonality,  and  linear  predictability  constraints  that  limit  the  codes 
which  can  be  stably  learned  by  alternative  recognition  models. 
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SPEECH  PERCEPTION  AND  PRODUCTION 
BY  A  SELF-ORGANIZING  NEURAL  NETWORK 
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Y.C.  Lee  (Ed.),  Hong  Kong:  World  Scientific,  1988 

ABSTRACT 

Considerations  of  the  real-time  self-organization  of  neural  networks  for  speech  recog¬ 
nition  and  production  have  lead  to  a  new  understanding  of  several  key  issues  in  such 
networks,  most  notably  a  definition  of  new  processing  units  and  functions  of  hierarchical 
levels  in  the  auditory  system.  An  important  function  of  a  particular  neural  level  in  the 
auditory  system  is  to  provide  a  partially-compressed  code,  mapped  to  the  articulatory  sys¬ 
tem,  to  permit  imitation  of  novel  sounds.  Furthermore,  top-down  priming  signals  from  the 
articulatory  system  to  the  auditory  system  help  to  stabilize  the  emerging  auditory  code. 
These  structures  help  explain  results  from  the  motor  theory,  which  states  that  speech  is 
analyzed  by  how  it  would  be  produced.  Higher  stages  of  processing  require  chunking  or 
unitization  of  the  emerging  language  code,  an  example  of  a  classical  grouping  problem. 
The  partially  compressed  auditory  codes  axe  further  compressed  into  item  codes  (e.g., 
phonemic  segments),  which  are  stored  in  a  working  memory  representation  whose  short¬ 
term  memory  pattern  is  its  code.  A  masking  field  level  receives  input  from  this  working 
memory  and  encodes  this  input  into  list  chunks,  whose  top-down  signals  organize  the  items 
in  working  memory  into  coherent  groupings  with  invariant  properties.  This  total  archi¬ 
tecture  sheds  new  light  on  key  speech  issues  such  as  coarticulation,  analysis-by-synthesis, 
motor  theory,  categorical  perception,  invariant  speech  perception,  word  superiority,  and 
phonemic  restoration. 
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CONTENT-ADDRESSABLE  MEMORY  STORAGE 
BY  NEURAL  NETWORKS:  A  GENERAL  MODEL 
AND  GLOBAL  LIAPUNOV  METHOD 


Stephen  Grossbergf 

In  E.L.  Schwartz  (Ed.),  Computational  Neuroscience 
Cambridge,  MA:  MIT  Press,  1987 

ABSTRACT 

Many  neural  network  models  capable  of  content-addressable  memory  are  shown  to 
be  special  cases  of  the  general  model  and  global  Liapunov  function  described  by  Co¬ 
hen  and  Grossberg  (1983).  These  include  examples  of  the  additive,  brain-state-in-a-box, 
McCulloch-Pitts,  Boltzmann  machine,  shunting,  masking  field,  bidirectional  associative 
memory,  Volterra-Lotka,  Gilpin- Ayala,  and  Eigen-Schuster  models.  The  Cohen-Grossberg 
model  thus  defines  a  general  principle  for  the  design  of  content-addressable  networks.  A 
model-independent  property,  such  as  content-addressable  memory,  that  is  shared  by  all 
model  exemplars  of  such  a  general  design  constitutes  a  computational  invariant.  Such 
a  general  model  and  analytic  method  defines  a  computational  framework  within  which 
specialized  model  exemplars  may  be  compared  to  discover  which  models  are  best  able 
to  explain  particular  parametric  data  about  brain  and  behavior,  or  to  solve  particular 
technological  problems. 


I 


t 


t  Supported  in  part  by  the  Air  Force  Office  of  Scientific  Research  (AFOSR  F49620- 
86-C-0037  and  AFOSR  F49620-87-C-0018)  and  the  National  Science  Foundation  (NSF 

IRI-84- 17756). 


> 


NONLINEAR  NEURAL  NETWORKS: 

PRINCIPLES,  MECHANISMS,  AND  ARCHITECTURES 

Stephen  Grossbergf 
Neural  Networks,  1988,  1,  17-61 
ABSTRACT 

An  historical  discussion  is  provided  of  the  intellectual  trends  that  caused  19th  cen¬ 
tury  interdisciplinary  studies  of  physics  and  psychobiology  by  leading  scientists  such  as 
Helmholtz,  Maxwell,  and  Mach  to  splinter  into  separate  20th  century  scientific  move¬ 
ments.  The  nonlinear,  nonstationary,  and  nonlocal  nature  of  behavioral  and  brain  data 
are  emphasized.  Three  sources  of  contemporary  neural  network  research — the  binary,  lin¬ 
ear,  and  continuous-nonlinear  models — are  noted.  The  remainder  of  the  article  describes 
results  about  continuous-nonlinear  models:  Many  models  of  content-addressable  mem¬ 
ory  axe  shown  to  be  special  cases  of  the  Cohen-Grossberg  model  and  global  Liapunov 
function,  including  the  additive,  brain-state-in-a-box,  McCulloch-Pitts,  Boltzmann  ma¬ 
chine,  Hartline-Ratliff-Miller,  shunting,  masking  field,  bidirectional  associative  memory, 
Volterra-Lotka,  Gilpin- Ayala,  and  Eigen-Schuster  models.  A  Liapunov  functional  method 
is  described  for  proving  global  limit  or  oscillation  theorems  for  nonlinear  competitive  sys¬ 
tems  when  their  decision  schemes  are  globally  consistent  or  inconsistent,  respectively.  The 
former  case  is  illustrated  by  a  model  of  a  globally  stable  economic  market,  and  the  latter 
case  is  illustrated  by  a  model  of  the  voting  paradox.  Key  properties  of  shunting  competi¬ 
tive  feedback  networks  are  summarized,  including  the  role  of  sigmoid  signalling,  automatic 
gain  control,  competitive  choice  and  quantization,  tunable  filtering,  total  activity  normal¬ 
ization,  and  noise  suppression  in  pattern  transformation  and  memory  storage  applications. 
Connections  to  models  of  competitive  learning,  vector  quantization,  and  categorical  per¬ 
ception  are  noted.  Adaptive  resonance  theory  (ART)  models  for  Belf-stabilizing  adaptive 
pattern  recognition  in  response  to  complex  real-time  nonstationary  input  environments  are 
compared  with  off-line  models  such  as  autoassociators,  the  Boltzmann  machine,  and  back 
propagation.  Special  attention  is  paid  to  the  stability  and  capacity  of  these  models,  and  to 
the  role  of  top-down  expectations  and  attentional  processing  in  the  active  regulation  of  both 
learning  and  fast  information  processing.  Models  whose  performance  and  learning  are  reg¬ 
ulated  by  internal  gating  and  matching  signals,  or  by  external  environmentally  generated 
error  signals,  are  contrasted  with  models  whose  learning  is  regulated  by  external  teacher 
signals  that  have  no  analog  in  natural  real-time  environments.  Examples  from  sensory- 
motor  control  of  adaptive  vector  encoders,  adaptive  coordinate  transformations,  adaptive 
gain  control  by  visual  error  signals,  and  automatic  generation  of  synchronous  multijoint 
movement  trajectories  illustrate  the  former  model  types.  Internal  matching  processes  are 
shown  capable  of  discovering  several  different  types  of  invariant  environmental  properties. 
These  include  ART  mechanisms  which  discover  recognition  invariants,  adaptive  vector 
encoder  mechanisms  which  discover  movement  invariants,  and  autoreceptive  associative 
mechanisms  which  discover  invariants  of  self-regulating  target  position  maps. 
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ABSTRACT 

A  real-time  neural  network  model,  called  Affective  Balance  Theory,  is  developed  to  ex¬ 
plain  many  properties  of  decision  making  under  risk  which  heretofore  have  been  analysed 
using  formal  algebraic  models,  notably  Prospect  Theory.  The  model  describes  cognitive- 
emotional  interactions  which  are  designed  to  ensure  adaptive  responses  to  environmental 
demands,  but  whose  emergent  properties  nonetheless  can  lead  to  paradoxical  and  even 
irrational  decisions  in  risky  environments.  Emotional  processing  in  the  model  is  carried 
out  by  an  opponent  processing  network,  called  a  gated  dipole.  Learning  enables  cognitive 
representations  to  generate  affective  reactions  of  the  dipole.  Habituating  chemical  trans¬ 
mitters  within  a  gated  dipole  determine  an  affective  adaptation  level,  or  context,  against 
which  later  events  are  evaluated.  Neutral  events  can  become  affectively  charged  either 
through  direct  activations  or  antagonistic  rebounds  within  a  previously  habituated  dipole. 
The  theory  describes  the  affective  consequences  of  strategies  in  which  an  individual  com¬ 
pares  pairs  of  events  or  statements  that  are  not  necessarily  explicitly  grouped  within  the 
stimuli.  The  same  preference  orders  may  sometimes,  but  not  always,  emerge  from  dif¬ 
ferent  sequences  of  pairwise  alternatives.  The  role  of  short  term  memory  updating  and 
expectancy-modulated  matching  processes  in  regulating  affective  reactions  is  described. 
The  formal  axioms  of  Prospect  Theory  are  dynamically  explicated  through  this  analy¬ 
sis.  Analyses  of  judgments  of  the  utility  of  a  single  alternative,  choices  between  pairs  of 
regular  alternatives,  choices  between  riskless  and  risky  alternatives,  and  choices  between 
pairs  of  risky  alternatives  lead  to  explanations  of  such  phenomena  as  preference  reversals, 
gambler’s  fallacy,  framing  effect,  and  the  tendency  towards  risk  aversion  when  gains  are 
involved  but  risk  taking  when  losses  are  involved.  These  explanations  illustrate  that  data 
concerning  decision  making  under  risk  may  now  be  related  to  data  concerning  the  dynam¬ 
ics  of  conditioning,  cognition,  and  emotion  as  consequences  of  a  single  psychophysiological 
theory. 
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AND  SECONDARY  REINFORCEMENT 
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ABSTRACT 

Selective  information  processing  in  neural  networks  is  studied  through  computer  sim¬ 
ulations  of  Pavlovian  conditioning  data.  The  model  reproduces  properties  of  blocking, 
inverted-U  in  learning  as  a  function  of  interstimulus  interval,  anticipatory  conditioned  re¬ 
sponses,  secondary  reinforcement,  attentional  focussing  by  conditioned  motivational  feed¬ 
back,  and  limited  capacity  short-term  memory  parocessing.  Conditioning  occurs  from 
sensory  to  drive  representations  (“conditioned  reinforcer”  learning),  from  drive  to  sensory 
representations  (“incentive  motivational”  learning),  and  from  sensory  to  motor  represen¬ 
tations  (“habit”  learning).  The  conditionable  pathways  contain  long-term  memory  traces 
that  obey  a  non-Hebbian  associative  law.  The  neural  model  embodies  a  solution  to  two 
key  design  problems  of  conditioning,  the  synchronization  and  persistence  problems.  This 
model  of  ertebrate  learning  is  compared  with  data  and  models  of  invertebrate  learning. 
Predictions  derived  from  models  of  vertebrate  learning  are  compared  with  data  about  in¬ 
vertebrate  learning,  including  data  from  Aplyaia  about  facilitator  neurons  and  data  from 
Hermisacnda  about  voltage-dependent  Ca++  currents.  A  prediction  is  stated  about  clas¬ 
sical  conditioning  in  all  species,  called  the  Secondary  Conditioning  Alternative,  and  if 
confirmed  would  constitute  an  evolutionary  invariant  of  learning. 
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ABSTRACT 

A  real-time  neural  network  model  is  described  in  which  reinforcement  helps  to  focus 
attention  upon  and  organize  learning  of  those  environmental  events  and  contingencies  that 
have  predicted  behavioral  success  in  the  past.  Computer  simulations  of  the  model  repro¬ 
duce  properties  of  attentional  blocking,  inverted-U  in  learning  as  a  function  of  interstim¬ 
ulus  interval,  primary  and  secondary  excitatory  and  inhibitory  conditioning,  anticipatory 
conditioned  responses,  attentional  focussing  by  conditioned  motivational  feedback,  and 
limited  capacity  short  term  memory  processing.  Qualitative  explanations  are  offered  of 
why  conditioned  responses  extinguish  when  a  conditioned  excitor  is  presented  alone,  but 
do  not  extinguish  when  a  conditioned  inhibitor  is  presented  alone.  These  explanations 
invoke  associative  learning  between  sensory  representations  and  drive,  or  emotional,  rep¬ 
resentations  (in  the  form  of  conditioned  reinforcer  and  incentive  motivational  learning), 
between  sensory  representations  and  learned  expectations  of  future  sensory  events,  and 
between  sensory  representations  and  learned  motor  commands.  Drive  representations  are 
organized  in  opponent  positive  and  negative  pairs  (e.g.,  fear  and  relief),  linked  together 
by  recurrent  gated  dipole,  or  READ,  circuits.  Cognitive  modulation  of  conditioning  is 
regulated  by  adaptive  resonance  theory,  or  ART,  circuits  which  control  the  learning  and 
matching  of  expectations,  and  the  match-contingent  reset  of  sensory  short  term  memory. 
Dendritic  spines  are  invoked  to  dissociate  read-in  and  read-out  of  associative  learning  and 
to  thereby  design  a  memory  which  does  not  passively  decay,  does  not  saturate,  and  can  be 
actively  extinguished  by  opponent  interactions. 
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ABSTRACT 

A  neural  network  model  of  multiple-scale  binocular  fusion  and  rivalry  in  visual  cor¬ 
tex  is  described  and  simulated  on  the  computer.  The  model  consists  of  three  parts:  a 
distributed  spatial  representation  of  binocular  input  patterns  among  simple  cells  that  are 
organized  into  ocular  dominance  columns;  an  adaptive  filter  from  simple  cells  to  complex 
cells;  and  a  nonlinear  on-center  off-surround  shunting  feedback  network  that  joins  together 
the  complex  cells.  This  data  structure  generates  complex  cell  receptive  fields  which  mul¬ 
tiplex  input  position,  orientation,  spatial  frequency,  positional  disparity,  and  orientational 
disparity,  and  which  are  insensitive  to  direction-of-contrast  in  the  image.  Multiple  copies  of 
this  circuit  are  replicated  in  the  model  using  receptive  fields  of  different  sizes.  Within  each 
such  circuit,  the  simple  cell  and  complex  cell  receptive  field  sizes  covary.  Together  these 
circuits  define  a  self-similar  multiple-scale  network.  The  self-similarity  property  across  spa¬ 
tial  scales  enables  the  network  to  exhibit  a  size-disparity  correlation,  whereby  simultaneous 
binocular  fusion  and  rivalry  can  occur  among  the  spatial  scales  corresponding  to  a  given 
retinal  region.  It  is  shown  that  a  laminar  organization  of  the  model  interactions  among 
the  complex  cells  gives  rise  to  conceptually  simple  growth  rules  for  intercellular  connec¬ 
tions.  The  output  patterns  of  the  model  complex  cells  are  designed  to  feed  into  the  model 
hypercomplex  cells  at  the  first  competitive  stage  of  a  Boundary  Contour  System  network, 
where  they  trigger  a  process  of  multiple-scale  emergent  binocular  boundary  segmentation. 
The  modelling  results  are  compared  with  psychophysical  data  about  binocular  fusion  and 
rivalry,  as  well  as  with  the  cepstrum  stereo  model  of  Yeshurun  and  Schwartz.  The  results 
indicate  that  analogous  self-similar  multiple-scale  neural  networks  may  be  used  to  carry 
out  data  fusion  of  many  other  types  of  spatially  organized  data  structures. 
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NEURAL  DYNAMICS  OF  SURFACE  PERCEPTION: 
BOUNDARY  WEBS,  ILLUMIN AN TS ,  AND  SHAPE-FROM-SHADING 
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ABSTRACT 

A  real-time  visual  processing  theory  is  used  to  provide  a  new  approach  to  the  analysis 
of  surface  perception,  notably  shape-from-shading.  The  theory  has  elsewhere  been  used 
to  explain  data  about  boundary  detection  and  completion,  textural  segmentation,  depth 
perception,  color  and  brightness  perception,  and  striate-prestriate  cortical  interactions. 
Neural  network  interactions  within  a  multiple  scale  Boundary  Contour  (BC)  System  and 
Feature  Contour  (FC)  System  are  used  to  explain  these  phenomena.  Each  spatial  scale 
of  the  BC  System  contains  a  hierarchy  of  orientationally  tuned  interactions,  which  can 
be  divided  into  two  successive  subsystems  called  the  OC  Filter  and  the  CC  Loop.  The 
OC  Filter  contains  two  successive  stages  of  oriented  receptive  fields  which  are  sensitive  to 
different  properties  of  image  contrasts.  The  OC  Filter  generates  inputs  to  the  CC  Loop, 
which  contains  successive  stages  of  spatially  short-range  competitive  interactions  and  spa¬ 
tially  long-range  cooperative  interactions.  Feedback  between  the  competitive  and  cooper¬ 
ative  stages  synthesizes  a  coherent,  multiple  scale  structural  representation  of  a  smoothly 
shaded  image,  called  a  boundary  web.  Such  a  boundary  web  regulates  multiple-scale  filling- 
in  reactions  within  the  FC  System  which  generate  a  percept  of  forzn-and-color-in-depth. 
Computer  simulations  establish  key  properties  of  a  boundary  web  representation:  nesting 
of  boundary  web  reactions  across  spatial  scales,  coherent  completion  and  regularization  of 
boundary  webs  across  incomplete  image  data,  and  relative  insensitivity  of  boundary  webs 
to  illumination  level  and  highlights.  The  theory  clarifies  data  about  interactions  between 
brightness  and  depth  percepts,  transparency,  influences  of  highlights  on  perceived  surface 
glossiness,  and  shape-from-texture  gradients.  The  total  network  suggests  a  new  approach 
to  the  design  of  computer  vision  systems,  and  promises  to  provide  a  universal  set  of  rules 
for  3-D  perceptual  grouping  of  scenic  edges,  textures,  and  smoothly  shaded  regions. 
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ABSTRACT 

A  real-time  neural  network  model  is  developed  to  explain  data  about  the  acquisition 
and  extinction  of  conditioned  excitors  and  inhibitors.  Systematic  computer  simulations  are 
described  of  a  READ  circuit,  which  joins  together  a  mechanism  of  associative  learning  with 
an  opponent  processing  circuit,  called  a  recurrent  gated  dipole.  READ  circuit  properties 
clarify  how  positive  and  negative  reinforcers  are  learned  and  extinguished  during  primary 
and  secondary  conditioning.  Habituating  chemical  transmitters  within  a  gated  dipole  de¬ 
termine  an  affective  adaptation  level,  or  context,  against  which  later  events  are  evaluated. 
Neutral  CS’s  can  become  reinforcers  by  being  associated  either  with  direct  activations  or 
with  antagonistic  rebounds  within  a  previously  habituated  dipole.  Neural  mechanisms  are 
characterized  whereby  conditioning  can  be  actively  extinguished,  and  associative  satura¬ 
tion  prevented,  by  a  process  called  opponent  extinction ,  even  if  no  passive  memory  decay 
occurs.  Opponent  extinction  exploits  a  functional  dissociation  between  read-in  and  read¬ 
out  of  associative  memory,  which  may  be  achieved  by  locating  the  associative  mechanism 
at  dendritic  spines.  READ  circuit  mechanisms  are  joined  to  cognitive-emotional  mecha¬ 
nisms  for  associative  learning  of  conditioned  reinforcers  and  of  incentive  motivation;  and  to 
cognitive,  in  particular  adaptive  resonance  theory,  mechanisms  for  activating  and  storing 
internal  representations  of  sensory  cues  in  a  limited  capacity  short  term  memory  (STM); 
for  learning,  matching,  and  mismatching  sensory  expectancies,  leading  to  the  enhancement 
or  updating  of  STM;  and  for  shifting  the  focus  of  attention  towards  sensory  representations 
whose  reinforcement  history  is  consistent  with  momentary  appetitive  requirements.  This 
total  neural  architecture  is  used  to  explain  conditioning  and  extinction  of  a  conditioned 
excitor;  conditioning  and  non-extinction  of  a  conditioned  inhibitor;  properties  of  condi¬ 
tioned  inhibition  as  a  “slave”  process  and  as  a  “comparator”  process,  including  effects  of 
pretest  deflation  or  inflation  of  the  conditioning  context,  of  familiar  or  novel  training  or  test 
contexts,  of  weak  or  strong  shocks,  and  of  preconditioning  US-alone  exposures.  The  same 
mechanisms  have  elsewhere  been  used  to  explain  phenomena  such  as  blocking,  unblock¬ 
ing,  overshadowing,  latent  inhibition,  superconditioning,  inverted  U  in  conditioning  as  a 
function  of  interstimlus  interval,  anticipatory  conditioned  responses,  partial  reinforcement 
acquisition  effect,  learned  helplessness,  and  vicious-circle  behavior.  The  theory  clarifies 
why  alternative  models  have  been  unable  to  explain  an  equally  large  data  base. 
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ABSTRACT 

A  neural  network  model  that  controls  behavioral  timing  is  described  and  simulated. 
This  model,  called  the  Spectral  Timing  Model,  controls  a  type  of  timing  whereby  an  animal 
or  robot  can  learn  to  wait  for  an  expected  goal  by  discounting  expected  nonoccurrences 
of  a  goal  object  until  the  expected  time  of  arrival  of  the  goal.  If  the  goal  object  does  not 
then  materialize,  the  animal  can  respond  to  unexpected  nonoccurrences  of  the  goal  with 
appropriate  changes  in  information  processing  and  exploratory  behavior.  The  model  is  a 
variant  of  the  gated  dipole  model  of  opponent  processing.  When  the  gated  dipole  model 
is  generalized  to  include  a  spectrum  of  cellular  response  rates  within  a  large  population 
of  cells,  the  model’s  total  output  signal  generates  accurate  learned  timing  properties  that 
collectively  provide  a  good  quantitative  fit  to  animal  learning  data.  In  particular,  the 
Spectral  Timing  Model  utilizes  the  habituative  transmitter  gates  and  adaptive  long  term 
memory  traces  that  are  characteristic  of  gated  dipole  models.  The  Spectral  Timing  Model 
is  embedded  into  an  Adaptive  Resonance  Theory  (ART)  neural  architecture  for  the  learning 
of  correlations  between  internal  representations  of  recognition  codes  and  reinforcement 
codes.  This  type  of  learning  is  called  conditioned  reinforcer  learning.  The  two  types  of 
internal  representations  are  called  sensory  representations  (5)  and  drive  representations 
[D).  Activation  of  a  drive  representation  D  by  the  Spectral  Timing  Model  inhibits  output 
signals  from  the  orienting  subsystem  (A)  of  the  ART  architecture  and  activates  a  motor 
response.  The  inhibitory  pathway  helps  to  prevent  spurious  resets  of  short  term  memory, 
forgetting,  and  orienting  responses  from  being  caused  by  events  other  than  the  goal  object 
prior  to  the  expected  arrival  time  of  the  goal.  Simulated  data  properties  include  the 
inverted  U  in  learning  as  a  function  of  the  interstimulus  interval  (ISI)  that  occurs  between 
onset  of  the  conditioned  stimulus  (CSl  and  the  unconditioned  stimulus  (US);  correlations 
of  peak  time,  standard  deviation,  WeDer  fraction,  and  peak  amplitude  of  the  conditioned 
response  as  a  function  of  the  ISI;  increase  of  conditioned  response  amplitude,  but  not  its 
timing,  with  US  intensity;  speed-up  of  the  timing  circuit  by  an  increase  in  CS  intensity  or 
by  drugs  that  increase  concentrations  of  brain  dopamine  or  acetylcholine;  multiple  timing 
peaks  in  response  to  learning  conditions  using  multiple  ISI’s;  and  conditioned  timing  of  cell 
activation  within  the  hippocampus  and  of  the  contingent  negative  variation  (CNV)  event- 
related  potential.  The  results  on  speed-up  by  drugs  that  increase  brain  concentrations  of 
dopamine  and  acetylcholine  support  a  1972  prediction  that  the  gated  dipole  habituative 
transmitter  is  a  catecholamine  and  its  long  term  memory  trace  transmitter  is  acetylcholine. 
It  is  noted  that  the  timing  circuit  described  herein  is  only  one  of  several  functionally  distinct 
neural  circuits  for  governing  different  types  of  timed  behavior  competence. 
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